A Markerless Motion Tracking Approach To Understand Changes In Gait And Balance: A Case Study

Elham Dolatabadi1,2, Babak Taati1,2, Gemma S. Parra-Dominguez2 and Alex Mihailidis1,2

University of Toronto1; Toronto Rehabilitation Institute2

ABSTRACT     

The demand for a home-based, cost-effective, quantitative, and continuous assessment of balance is growing in order to better understand changes in gait and balance. This paper presents a case study on the feasibility of using a low cost and portable motion capture system (a Microsoft Kinect) for detecting changes in gait and movement patterns of older adults with balance impairment following early discharge from the hospital. The promising results of the study show that the Kinect has the potential to be integrated into a patient's home and monitor changes in spatio-temporal gait parameters during the recovery period.

INTRODUCTION

Balance and gait in the older adult population need special attention since they affect their functional mobility and safety (Gertenbach, 2002). Previous studies have shown that most older adults report some difficulties with balance and these difficulties associate with falls in this age group (Hausdorff, Rios, & Edelberg, 2001). Consequently, there is a need for measures of gait to monitor the recovery status of older adults over time and to assess their balance. On the other hand, typical gait analysis used in the clinic or lab cannot reflect long term changes in balance since balance failures often take place in homes, where standardized assessment cannot be applied. Moreover, tasks performed in the gait laboratory or clinics do not always simulate normal daily activities of older adult as people may feel more distracted at home than in the laboratory.

A gait laboratory typically uses different tools including force platform, pressure sensing walkway, and 3D motion analysis to evaluate the biomechanics of gait (Tyson & Souza, 2002; Kejonen, Kauranen, Ahasan, & Vanharanta, 2002). However, despite their high precision, these laboratory tools are less appropriate for home use due to their high cost and elaborate setup requirements. These facts motivate the assessment of gait characteristics in an older adult’s own home through the analysis of functional tasks.

BACKGROUND

With recent advances in technology and motion sensors, new possibilities for long term objective measures of gait and balance at home have started to come to light. Several research groups are already developing gait assessment tools using the wearable sensor including accelerometers and gyroscopes (Najafi, et al., 2003; Tognetti, et al., 2005). Although these systems are small and portable, they suffer from several limitations such as short battery life, not capturing information about the environmental context, and sensitivity to sensor placement e.g. foot vs. ankle.

Over the past decade, vision-based human gait analysis has continued to be a thriving area of research. Markerless motion capture systems using Web-Cameras offer an opportunity to reconstruct kinematic features comparable to gait laboratory tools and motion sensors (Allin, Beach, Mitz, & Mihailidis, 2008; Snoek, Hoey, Stewart, Zemel, & Mihailidis, 2009). These systems do not require the placement of additional sensors or markers on or around the patient's body. However, they do require a controlled environment and a large number of cameras as they rely on the processing of monocular vision. Moreover, they generally need complicated algorithms in order to fit a skeletal model to the silhouettes.

Depth sensors such as the Microsoft Kinect offer several advantages over Web-Cameras including working in low light or dark environments and simplifying the task of silhouettes extraction. The accuracy of using Kinect for mobility assessment in comparison with 3D motion capture (VICON) and Web-Camera have been evaluated by a few research groups (Dutta, 2012; Gabel, Renshaw, Schuster, & Gilad-Bachrach, 2012).

In terms of quantitative analysis of gait using the Kinect, a majority of previous research focused on only measuring the gait parameters in a clinical or laboratory setting. By contrast, our study explores the understanding of changes in gait in the home and during the recovery period to investigate the integration of the monitoring system into a real home environment.

OBJECTIVE

In order to determine the feasibility of capturing the balance parameters in the home and inferring the kinematics of functional activity recorded over time, we conducted a case study. For this case study, we have focused our objective on the total hip replacement (THR) population. The goal of this study was to unobtrusively perform quantitative analysis of the gait parameters in the home via an affordable low cost markerless motion capture.

SYSTEM DESCRIPTION

fillerFigure 1 : The deployment of Kinect sensor in the HomeLab.

Our system includes the Microsoft Kinect, which was deployed in the home to capture walking strategies (shown in Figure 1 ). We developed a skeletal tracking application based on the Microsoft Software Development Kit (SDK). The application tracks and records the 3D locations, i.e., the x, y, and z coordinate relative to the depth sensor, of 20 body joints. The program also records color and depth video streams. All recorded skeletal information and video frames are time stamped for post analysis of joint trajectories over time. We follow the displacement of each 3D joint location over time in order to extract primary kinematic features to understand changes in gait and balance.

METHOD

Data Collection

A 64 year old male with THR surgery on his right side participated in our case study. His participation began one day before the surgery and resumed subsequently one week after the surgery as soon as minimum recovery had taken place. Data collection continued once every one or two weeks throughout 9 weeks following the surgery at two different locations: (1) the participant's home for the first two sessions of recording after the surgery; and (2) the HomeLab (HomeLab, 2011) when the subject was able to come to the lab. The participant was instructed to walk forward toward the Kinect sensor at his preferred speed. He wore his own shoes and used crutches for 1 week and a cane for 2 weeks after the surgery. Each test was repeated 9 times which resulted in recording of 63 trials.

It should be noted that the practical range of Kinect is limited and it starts recognizing people standing at distances between 0.8 to 2.5 meters from the camera. Therefore, the beginning and end of each test sequence was selected to be 0.2 and 4 meters away from the camera respectively so the entire useful range of view was kept for the processing.

Preprocessing

As a first step, we computed the Centre of Mass (CoM) as the average 3D location of the hip, the shoulders, and the spine joints at each frame. The frames in the beginning and at the end of each test sequence were excluded if the computed CoM was beyond the practical viewing range of Kinect. This process cleaned up the data by discarding potentially erroneous skeletal tracking information recorded when the subject was outside the working range of the sensor.

Feature Extraction

Several movement features were computed from recorded joint positions via the processing of 3D skeletal sequence. Basic spatio-temporal gait parameters including the step length and the stance time on each side and the stride length and the cadence were measured.

We estimated the basic gait parameters based on the inspection of lower extremity 3D joint motions (specifically the ankle joint) along the z-axis (depth). During walking, when a foot is in a stance phase, its location should not be changing. By calculating the numerical derivative, we could therefore detect if a foot was in a swing phase (i.e., the location of the foot was changing) or was in a stance phase.

Statistical Analysis

In order to compare changes in gait characteristics between different stages ranging from pre-operative to 9 weeks post-operative, one-way analysis of variance (ANOVAs) were used. The analysis was conducted on all gait parameters including the stance time, the step length, the stride length, and the cadence measures. Using one-way ANOVAs, we compared the means of each gait parameter across the 7 time groups (1 day pre-surgery, 1, 2, 4, 6, 7 and 9 weeks post-surgery) to determine if the gait parameters improved with the stage post-operative. We also used the multiple comparison procedure to study which pairs of gait values were significantly different (p<0.05). According to this procedure, a significant difference between pairs can be determined if a 95% confidence interval for the true difference of their means does not contain 0.0.

RESULTS

fillerFigure 2 : The changes in the step length and the stance time on each side, the stride length andthe cadence across the spectrum of recovery from 1 day before the surgery to 9 weeks after the surgery.

The changes in the step length and the stance time on each side, the cadence and the stride length across the spectrum of recovery, from pre-operative to 9 weeks post-operative assessment, are all illustrated in Figure 2. The mean values of these gait parameters for each session are also shown in Figure 2. The most important results of one-way ANOVA and multiple comparison procedures are presented in Table 1.

One-way ANOVAs computed on each gait measure indicated that differences between the means of the 7 time groups were highly significant (see Table 1). We observed significant differences between the means of all time groups from pre-operative to all post-operative data points for all gait parameters. Accordingly, in order to determine which means significantly differed at the 0.05 level, we used the multiple comparison procedure. Between the pre-operative and each of the post-operative test measures, the most variation can be seen between one week pre-surgery and one week post-surgery for all measured gait parameters except the step length on the right side.

The step length measured pre-operatively on the operated side (right side) was significantly smaller than the one measured post-operatively by 9 weeks after the surgery. This difference shows the considerable improvement obtained within this period. However this pattern of change cannot be seen for the step length measured on the non-operated side (left side). Post-operatively, we observed significant difference on right step length and left stance time between 6 weeks and 9 weeks after the surgery. The stride length and cadence, by contrast, did not differ significantly throughout this period. 

Based on the results, gait characteristics improved significantly from 1 week to 6 weeks following the surgery and they all remained stable between the 6-week and 9-week points after the surgery. No significant differences were observed between the 1-week and 2-week points following the surgery for all gait measures. In terms of symmetry, except for the pattern of changes for step length, we did not observe any significant difference between the operated side and non-operated side.

DISCUSSION AND CONCLUSION

The aim of this study was to investigate the feasibility of using the Microsoft Kinect sensor to observe gait recovery after THR in the home. We presented a case study of integrating the Kinect into a participant’s home to track changes in step length, stride length, stance time and cadence through analysis of walking task for 1 day pre-operative and 9 weeks post-operative. Overall all measures revealed post-operative improvement in function following the surgery.

In brief, the measured gait parameters were all worsened in comparison with pre-operative values after the surgery and all started to improve from 1 week to 6 weeks following the surgery. Subsequently, they all remained stable as seen between the 6-week and 9-week points after the surgery except the step length.

Table 1 : Results of one-way ANOVA and multiple comparison procedure measured on each spatio-temporal gait parameter. (F denotes the F test statistic, P denotes the probability value, MS denotes the Mean square, Δ denotes the estimated difference between each pair in means and the square brackets denote the 95% confidence interval for the true difference of the means.)

 

All  time groups

pre &

1 wk

1 wk &

2 wks

6 wks &

9 wks

Step length

L

F=6.09

P=1e-04

MS= .04

 = .14

Δ = -.05

[-.19:.07]

Δ = -.003

[-.13:.14]

R

F=12.28

P=1e-08

MS= .06

Δ = .09

[-.01:.19]

Δ = -.007

[-.09:.1]

Δ = -.1

[-.2:-.003]

Stance time

L

F=7.63

P=1e-05

MS= .16

Δ = -.32

[-.53:-.11]

Δ = .05

[-.15:.3]

Δ = -.02

[-.2:-.18]

R

F=10.84

P=1e-06

MS= .13

Δ = -.3

[-.47:-.13]

Δ= .08

[-.08:.2]

Δ =.04

[-.12:.21]

Stride

F=17.59

P=1e-10

MS= .18

Δ = .24

[.08:.4]

Δ = - .03

[-.19:.12]

Δ = -0.11

[-.3:.04]

Cadence

F=20.52

P=1e-10

MS= 1e3

Δ =35.42

[21:49.5]

Δ = -7.6

[-21:6]

Δ = 0.7

[-13:10]

Moreover, we could not observe any significant difference between the left and the right side on the gait parameters during the study period. This is in agreement with previous studies (Mikia, et al., 2004; Wal, Ashburn, & Klenerman, 1981). The only asymmetrical feature was the difference between the patterns of changes in the step length measured on the right side (the operated side) compared with the one measured on left side. As the patient underwent the THR on his right side, the right step length measures improved throughout the 9 weeks following the surgery. The left step length measures also returned to the pre-operative baseline during this period.

In conclusion, we presented a case study in this paper which demonstrated the feasibility of using a low cost and markerless tool such as the Kinect to perform quantitative analysis of gait parameters in homes through observing functional tasks. Although the results of this case study are promising, they are conducted based on only one participant.

In the future, we hope to recruit more participants and run the study for longer follow-up periods which will strengthen the results.

REFERENCES

Allin, S., Beach, C., Mitz, A., & Mihailidis, A. (2008). Video based analysis of standing balance in a community center. Conf Proc IEEE Eng Med Biol Soc, (pp. 4531-4).

Dutta, T. (2012). Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace. Appl Ergo, 645-9.

Gabel, M., Renshaw, E., Schuster, A., & Gilad-Bachrach, R. (2012). Full Body Gait Analysis with Kinect. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

Gertenbach, H. J. (2002). The Influence of Proprioceptive Training on the Functional Balance of Older Adults. University of Stellenbosch.

Hausdorff, J., Rios, D., & Edelberg, H. (2001). Gait variability and fall risk in community-living older adults: a 1-year prospective study. Archives of Physical Medicine and Rehabilitation, 82(8), 1050-1056.

HomeLab. (2011). Retrieved from iDAPT Centre for Rehabilitation Research: http://www.torontorehab.com/Research/Facilities/Labs/HomeLab.aspx

Kejonen, P., Kauranen, K., Ahasan, R., & Vanharanta, H. (2002). Motion analysis measurements of body movements during standing: association with age and sex. Int J Rehabil Res, 25(4), 297-304.

Mikia, H., Suganoa, N., Hagioa, K., Nishiia, T., Kawakamia, H., Kakimotob, A., et al. (2004). Recovery of walking speed and symmetrical movement of the pelvis and lower extremity joints after unilateral THA. Journal of Biomechanics, 443–455.

Najafi, B., Aminian, K., Paraschiv-Ionescu, A., Loew, F., Bula, C. J., & Robert, P. (2003). Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Transactions on Biomedical Engineering, 50(6), 711-23.

Snoek, J., Hoey, J., Stewart, L., Zemel, R. S., & Mihailidis, A. (2009). Automated detection of unusual events on stairs. Journal of Image and Vision Computing, 27(1-2), 153-166.

Tognetti, A., Lorussi, F., Bartalesi, R., Quaglini, S., Tesconi, M., Zupone1, G., et al. (2005). Wearable kinesthetic system for capturing and classifying upper limb gesture in post-stroke rehabiltion. Journal of Neuroengineering and Rehabiliation, 2(8).

Tyson, S., & Souza, L. D. (2002). The measurement of balance post-stroke. Part 3: Instrumented measurement tools. Physical Therapy Reviews, 7(4), 231-241.

Wal, J., Ashburn, A., & Klenerman, L. (1981). Gait analysis in the assessment of functional performance before andafter total hip replacement. Journal of Biomedical Engineering, 121–127.